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Abstract 

Scientific modeling is a critical component of the scientific endeavor. Therefore, students 
who take science should learn about and understand the nature of this process. However, there has 
been little evidence that students understand the modeling process even in the increasing number 
of classrooms that use computer models in significant ways. This paper reports on the evaluation 
of an eleven-week curriculum that we created in order to foster seventh grade students’ 
understanding of scientific modeling. In the curriculum, students engaged in model-oriented 
activities such as creating non-Newtonian computer microworlds to embody their conceptual 
models, evaluating their models with criteria, and reflecting on the nature of models. This paper 
discusses various methods of assessing modeling as well as the results from the curriculum 
trials. Overall analysis suggests that students gained a significantly better understanding of the 
nature and utility of models without promoting similar gains in students’ understanding of the 
process of creating and evaluating of models. These results indicate that while modeling 
knowledge is difficult to obtain even from an extended model-focused curricula, progress can be 
made by further refining model-oriented curricula and assessments. 
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Introduction 

During the past several decades, computer modeling and simulation software have 
transformed the nature of science and engineering. These tools have enabled scientists and 
engineers to create, test, and envision their theories as well as analyze data in ways never before 
possible. Reformers in education have envisioned a similar technology-led transformation in 
education. In particular, numerous researchers have promoted the use of computer modeling and 
simulation tools in order to foster student conceptual change (Lewis, Rader, & Brand, 1997; 
Mandinach & Cline, 1992; Mellar, Bliss, Boohan, Ogbom & Tompsett, 1994; Penner, Giles, 

Lehrer, & Schauble, 1997; Raghavan & Glaser, 1995; Richards, Barowy and Levin, 1992; Snir, 
Smith and Grosslight, 1995; Stewart, Hafner & Johnson, 1992; Tinker, 1993; White, 1993; White & 
Frederiksen, 1990; White & Frederiksen, 1998). 

While these tools have shown great promise for their use in conceptual change, our 
research as well as that of others (Carey & Smith, 1993; Grosslight, Unger, Jay, & Smith, 1991, 
Schwarz, 1996) has found that few students understand the scientific modeling process 1 embodied 
in these curricula with which they are engaged. For example, we found that although students in 
the previous version of the ThinkerTools scientific inquiry and modeling curriculum were 
generally successful at creating laws to summarize their experimental results, few understood the 
nature or purpose of the modeling portion of our scientific inquiry cycle. Other research confirms 
that a modeling perspective is difficult to obtain in the classroom even from curricula with an 
added focus on the nature of modeling. 

Teaching students about the nature of scientific models 2 and the process of modeling is 
important for a variety of reasons. Firstly, models and the process of modeling are fundamentally 
important components of the scientific endeavor. Students taking science should know about the 
nature of the products and processes of science (Andaloro, Donzelli, Sperandeo-Mineo, 1991). 
Secondly, gaining a more authentic understanding of the nature of science may help students 
construct fruitful epistemological theories (Driver, Leach, Millar, & Scott, 1996; Gilbert, 1991; 
Penner, Giles, Lehrer, & Schauble, 1997; Nadeau, & Desautels, 1984). For example, if a student 
understands that scientific knowledge is complex and dynamic, this may help them better reason 
about scientific evidence and better integrate their conceptual knowledge (Songer & Linn, 1991). 
Additionally, using computer simulation models can afford students (as well as scientists) the 
opportunity to embody, visualize, and test components of their conceptual theories, which may help 
them advance those theories at the same time as gaining a better understanding of the process of 
model generation (Snir et al., 1995) Finally, technological advances and the resulting 
“information age” have changed the type of expertise that citizens need to function well in society. 
Matthews (1994), Sagan (1996), and others point out that we will need sophisticated and transferable 
inquiry and problem-solving skills to function in the communities and workplaces of the future. 
Teaching students about the nature of scientific models and the process of modeling while using 
computer modeling software may foster this type of transferable inquiry and problem-solving 
expertise. 

In order to address this important area of scientific expertise, we created the model- 
enhanced version of the ThinkerTools curriculum to foster an increased model-oriented 
perspective. In our curricula, students participate in activities that include (1) creating computer 
microworlds to embody their conceptual models of force and motion, (2) evaluating the accuracy 
and plausibility of their models with respect to data collected from real-world experiments, and (3) 
reflecting on the nature of the modeling enterprise itself. The curriculum ran in eight seventh 



1 We define scientific modeling as a process used in much of modem science and engineering that involves (1) embodying 

key aspects of a theory into a model- frequently a computer model, (2) testing that model, and (3) revising that model. 

2 For the purposes of our science curriculum, we define a scientific model very broadly as a set of representations, rules, and 

reasoning structures that allow one to generate predictions and explanations. 
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grade classrooms in a local urban middle school for approximately eleven weeks. The first author 
co-taught one class and was a teacher’s aide in another class at the school. The curriculum covered 
several content topics including one-dimensional motion with and without friction, two- 
dimensional motion, and either gravity, mass, or gas/fluid resistance depending on the students’ 
choice for a final research project. 

In evaluating the effectiveness of the curriculum, we investigated students’ knowledge 
about the nature of models and the process of modeling. This entailed studying several 
dimensions of student understanding (1) model content and attributes, (2) the nature of the 
modeling process, (3) the evaluation of models, and (4) the purpose of models and modeling. Data 
used to analyze these questions include written assessments before and after the curriculum, 
student project reports, and student interviews. 

We hypothesized that our students would improve their understanding of models and 
modeling by participating in some of the model-oriented activities in the curriculum. However, 
we were also aware of the difficulty of instantiating large changes in students’ knowledge of the 
nature of science from previous research (Carey & Smith, 1993). As a result, we estimated that 
students modeling knowledge after the model-enhanced ThinkerTools would vary between fairly 
unsophisticated understanding (similar to Grosslight et al.’s level 1+) to somewhat sophisticated 
understanding (similar to Grosslight et al.’s level 2) depending on aspects like the student’s 
propensity towards metacognition, their previous scientific experiences, and their teacher’s 
understanding of the scientific process. 



Background 

Research and interest about the potential benefits of model-centered science instruction 3 
has grown in the recent decade as science educators have seen computer modeling and simulation 
software transform the practices of science and engineering. For example, philosophers of science 
such as Gilbert (1991) and educators such as Nadeau and Desautels (1984) and Penner et al. (1997), 
espouse a model-centered approach in science classrooms in order to combat learning naive 
philosophies of science such as naive realism and empiricism. Scientists such as Andoloro et al. 
(1991) advocate using computer simulation in order to accurately reflect the methodology of much 
of modem science and physics. Authors such as Bliss (1994), Confrey and Doerr, (1994), Lewis, 
Rader, and Brand (1997), Papert (1980), and Rothenberg (1989) claim that computer models are a 
useful modeling tool and can help people externalize their thinking, thus making their theories 
concrete and allowing them to be tested, validated and refined. 

Educators claim further benefits. Feurzeig (1994) and Sabelli (1994) state that using 
computer models helps make real data more accessible to students, making the scientific process 
more dynamic, and allowing students to study personally interesting and complex phenomenon. 
Jackson, Stratford, Krajcik, and Soloway, 1994, Raghavan and Glaser (1995), Richards et al. 
(1992), Snir et al., 1993, Stewart et al. (1992), White and Horowitz (1988), White (1993), and White 
and Frederiksen (1995) advocate the instructional use of computer models in order to either have 
students create scientific artifacts, to foster student conceptual change, or to develop students’ 
scientific expertise. While these authors have studied and promoted the idea of using computer 
models and teaching general modeling in highly beneficial ways, few researchers, except some 
like White and Frederiksen (1995), and White (1993), have investigated students’ understandings 
of the scientific modeling process they are using. 



3 By model-centered science instruction, we mean instruction whose main focus is on using or on constructing and revising 
scientific models. 
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However, additional literature about the general role of epistemological beliefs in 
learning and school performance provides some information about students’ understanding of the 
scientific modeling process. While much of this literature covers other aspects of epistemological 
beliefs such as the nature of scientific knowledge or the nature of learning, some researchers have 
studied student beliefs about the nature of the scientific process including scientific modeling 
(Carey & Smith, 1993; Driver et al., 1996; Gobert & Discenna, 1997; Grosslight et al., 1991; Penner 
et al., 1997; Robinson, 1997). For example, Driver et al. (1996) studied British students’ ideas about 
their characterization of scientific inquiry, the nature and status of scientific theories, and their 
ideas on the relationship between theory and evidence. Other authors such as Carey and Smith 
(1993), and Grosslight, Unger, and Jay (1991) have studied students’ characterization of the nature 
of models and modeling in order to develop a three level scale of epistemological understanding. 
In their case, level 3, the targeted level of understanding, involves understanding that the model is 
constructed in the service of the development and testing of ideas, that the modeler takes an active 
role in constructing the model, and models can be manipulated and subjected to testing in the 
service of informing ideas. Finally, other authors have shown that specific beliefs about the nature 
of learning (Schommer et al., 1992) and the nature of physics knowledge (Hammer, 1994) correlate 
with student performance, while Linn and Songer (1993) and Songer and Linn (1991) showed that 
student beliefs about the static or dynamic nature of science directly influenced student knowledge 
integration- a process of organizing information into broader categories. 



Setting 



Experimental Design 



The model-enhanced ThinkerTools curriculum was used during the 1996-1997 academic 
year in eight seventh grade classes at a local urban middle school with a diverse ethnic population 
and a high proportion of lower income students (During the 1996-1997 academic year, 34% of the 
students qualified for free or reduced school lunch). As previously mentioned, the first author co- 
taught one class (Teacher A, first period), was a teacher’s aide in another (Teacher B, second 
period) (See table below for elaboration). The two teachers with which she collaborated (Teacher A 
and B) used the curriculum in their remaining six classes. The curriculum lasted approximately 
eleven weeks. 



Teacher 


Period 


Number of 
Students 


First author presence in the 
classroom 


Material covered 


A 


i 


30 


co-taught daily and videotaped 


Module 1, 2, 3, 4 


A 


3 


27 


was occasionally present to help 
and videotape 


Module 1, 2, 3, 4 


A 


4 


37 


not present 


Module 1, 2, 3, 4 


A 


5 


31 


not present 


Module 1, 2, 3, 4 


B 


1 


28 


not present 


Module 1, 3 (partly), 4 


B 


2 


25 


teacher’s aide and videotaped 
daily 


Module 1, 3 (partly), 4 


B 


j 3 


25 


not present 


Module 1, part of 3, 4 


B 


4 


26 


not present 


Module 1, part of 3, 4 



Table 1: Description of model-enhanced ThinkerTools classrooms 



Intervention Content 

We designed the model-enhanced version of ThinkerTools to teach students about the 
nature of models and the process of modeling. As previously mentioned, the main pedagogical 



ERIC 



AERA 1998, San Diego 



6 



page 5 



Scientific Modeling 



Christina Schwarz & Barbara White 



goals for student learning included: understanding (1) the nature of models (model content and 
attributes), (2) the nature of modeling, (3) how to evaluate models, and (4) the utility of modeling. 

In the model-enhanced version of ThinkerTools, the main types of modeling activities that 
addressed these pedagogical goals included creating computer microworlds governed by student 
rules about force and motion, evaluating models, and reflecting on the properties of models and the 
nature of the modeling enterprise. 

During the curriculum, students carried out research on force and motion by following the 
inquiry cycle four times. The inquiry cycle is a simplified but authentic version of the scientific 
method used in the ThinkerTools curriculum. (See figure 1 below) 



I Hypothesize | 



Question | 

t 

Evaluate 1 

\ 

| Model 



Investigate 

~T 

Analyze 

7 



Figure 1. The ThinkerTools scientific inquiry cycle 

Students researched four topics which included one-dimensional motion with and without friction, 
two-dimensional motion, and a research project on one of the following topics: gravity, mass, or 
gas/fluid resistance. In the first module, for example, students began with the question, "what is 
the motion of an object when no forces like friction are acting on it?” In the hypothesis phase of the 
inquiry cycle, students then made alternative hypotheses about the answer to this question such as 
the object would slow down, speed up, or travel at a constant speed. During the investigation phase 
of the inquiry cycle, students conducted real-world experiments such as giving a smooth plastic 
puck an impulse and measuring the object’s speed over one and two meters. They then reduced the 
amount of friction between the puck and the floor by adding a balloon and stopper to the puck, 
acting like a mini-hovercraft, and repeated the experiment, measuring the objects’ speed over one 
and two meters. In the analysis phase of the inquiry cycle, the students then analyzed the 
differences in speed between the first and second meters for the pucks with different amounts of 
friction. 



In the modeling phase of the inquiry cycle, the students analyzed their data and formed a 
tentative rule. In the first module of the curriculum, the students performed a thought experiment to 
create a tentative rule to characterize what would happen to the motion of an object with no friction. 
(In the three remaining modules, students created a rule derived from their real world data.) 
Students then went to the computer and chose the model that most closely corresponded to their rule 
including Newtonian and non-Newtonian rules. (See figure 2 below of the software screen shot) 
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= Model Design 



Options: Motion lOith No Force Like Friction 



3 



The motion of an object when no forces like friction are acting on it: 
O Slows Down 

An object in motion will slow down when there is no force like friction acting on it. The object 
will move slower and slower. 

O Speeds Up 

An object in motion will speed up when there is no force like friction acting on it. The object will 
move faster and faster . 

® Constant Speed 

An object in motion will remain in motion at a constant speed when there is no force like friction 
acting on it. 






Cancel ] j 



Done 



3 






Figure 2: This screen shot illustrates how the software enables students to choose from among 

alternative laws of motion. 



For example, if their data indicated that reducing the amount of friction caused an object to travel 
further and remain at approximately a constant speed, they might have chosen the Newtonian 
qualitative model “constant speed” to indicate that with no friction, an object would travel at a 
constant speed. Once students at the computer had chosen among three or four qualitative or semi- 
qualitative rules, they then typed in a causal or mechanistic explanation to the prompt, “I think this 
is true because:” Finally, the student ran a simulation which used their model in order to envision 
its consequences. Alternatively, they could have created an entirely new microworld with dots, 
walls, targets and other properties that used their models. (See figure 3 below for an example 
simulation from that model) 



BEST COPY AVAILABLE 
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Figure 3: A screen shot from a simulation activity in which students run an experiment and see 
the implications of the laws of motion they have selected. 



Allowing students to choose and envision their own models based on data from their 
experiments is the essence of scientific modeling. This activity was meant to address the 
pedagogical goals of teaching students about the nature of models (a model can be as simple as a 
rule that allows someone to explain and predict a phenomena; models are not necessarily real or 
correct but good ones are better estimates of a phenomenon; there are multiple models for the same 
phenomenon), the nature of modeling (modeling involves embodying key parts of a theory into a 
rule, among other aspects), and the utility of modeling (models are particularly useful for 
envisioning or testing that theory). Additionally, we designed the ThinkerTools software so that 
students could choose and explore different previously programmed models close to their own 
intuitions instead of programming their own intuitive-based models. We made this decision 
based on the idea that teaching students how to program in addition to the already long eleven week 
ThinkerTools curriculum was too time-consuming and impractical. 

There were several other software features designed to foster student learning about the 
nature of models and modeling. After students had run the simulation to test or envision their 
theory, they were encouraged to run their simulation in “single step mode.” During single step 
mode, the computer talked aloud about its behavior and modeling rule for each time step. For 
example, suppose that I chose the rule, “constant speed” about motion with no force like friction and 
responded to the “I think this is true because:” prompt by typing, “there are no forces applied to me to 
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change my motion.” While running the simulation in single step mode, after the initial impulse 
and at each major interval of time, the computer said aloud, “Without a force like friction, I will 
stay the same speed because there are no forces applied to me to change my motion.” We 
hypothesized that students would be highly motivated to use single step mode to listen to the rule 
embodied in their chosen models. Single step model also addressed the pedagogical goal of 
teaching students about how the computer model works. We hoped that they would learn that the 
computer steps through time and causes the objects on the screen to behave according to rules that 
may be programmed or chosen. 

The other main feature of the software was that students could run each simulation 
according to Newton's laws, called Newtonian model design. While students were still in the 
modeling phase of the inquiry cycle, and right after they have chosen single step mode, they were 
encouraged to compare the model they had chosen with the Newtonian model. We introduced he 
Newtonian model as the model proposed by Isaac Newton, “a famous physicist from the 17th 
century who invented important models of force and motion.” In this way, students had a chance to 
compare and observe the scientifically normative model without being told that this was the 
'correct' model which they must obtain. The process also addressed the pedagogical goals of 
understanding the importance of testing, comparing and contrasting models. 

During the modeling phase of the inquiry cycle, students also read and reflected on several 
reading passages about models and modeling. These sections, entitled “collaborative thinking 
about models” included three passages about what a scientific model is, how the ThinkerTools 
computer program worked, and the utility of computer models. For example, in the passage on 
what a model is, students compared and contrasted three different maps of the area around their 
school in order to discuss advantages and disadvantages of different kinds of representations. 
Students read the passages in pairs and summarized the content to each other in order to reflect 
about the nature of modeling. After everyone in the class had gotten a chance to read the passage, 
we had the class discuss student responses and ideas. 

Another reflection component of the modeling curriculum involved students watching a 
videotape of modern uses of computer simulation models. The video segment included a computer 
simulated tornado storm, a simulation of two galaxies colliding, some impulse-based simulations 
of objects moving on surfaces, and a short clip of the video animation movie “Toy Story.” After the 
students finished watching the video, they discussed the various computer simulations and their 
utility in society, addressing the pedagogical goal of the purpose of scientific modeling, 
particularly for visualizing phenomena and performing difficult experiments or experiments not 
otherwise possible. Watching and discussing the video allowed students to see important images 
of computer models while motivating them to understand scientific modeling. 

In the last stage of the inquiry cycle, evaluation, students evaluated their models with 
respect to modeling criteria such as accuracy and plausibility, applicability to other situations, 
and limitations. We chose four main criteria for characterizing good models: accuracy, plausible 
mechanism, utility, and consistency. For their evaluations, students gave their chosen model a 
score of one through five with respect to their model criteria which they justified with a written 
response as well as an occasional oral response to other members of the class. Evaluating models 
with criteria was a critical component of this curriculum. Philosophically, it is important that 
students understand that there a multiple scientific models, none of which entail absolute reality, 
and that some models are better than others with respect to the various criteria. Further, in order to 
make the modeling evaluation process similar to the evaluations within a scientific community, 
students compared and contrasted their model choice to other students' choices with respect to the 
evaluation criteria in class debates. 
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Data Coding. Collection, and Analysis 

Before describing our methods of data collection and analysis, we first describe our 
framework for assessing modeling knowledge. Our assessments have been framed to test four 
dimensions of student modeling knowledge. (For dimensions, see table 2 below.) First, did 
students learn about the nature of models? In other words, did students learn what a model is, what 
models represent, and that there can be different kinds of models for the same object? Secondly, 
did students learn and understand the process of modeling? Did they understand that the 
modeling process involves embodying key aspects of a theory into a model, testing that model, and 
revising that model? Thirdly, what did students understand about the evaluation of models? Did 
they learn that there are ways to evaluate models to determine whether one is better than another? 
Did they learn about criteria used to decide this? And finally, did students in the model-enhanced 
version of ThinkerTools learn about what models can be useful for and how scientists and 
students can use models in science? 



1. Nature of models/theories/explanations: 

Kinds of models and Model attributes: What is a model? What is the best definition of a 
model? What are some examples? What makes those examples models? Why is 
ThinkerTools a computer model? What does the modeling portion of the inquiry cycle 
mean? 

Model content: What do models represent? What are models modeling? (objects? 
systems?) 

Multiple models: Can there be different models for the same object/phenomena? (ex: 
different models of an atom) Can you build models of different theories? (This is 
related to whether knowledge is simple or complex.) 

Constructed nature of models: Do models represent absolute reality, are they useful 

constructions, or are they useless ideas that are relative? (This is related to the nature 
of modeling category and addresses students' philosophy of science perspectives of 
realism , relativism , and constructivism.) Can you build a model of an incorrect 
theory? If a computer model doesn't behave like the real world, why do you think that 
is? 

2. Nature of modeling/inquiry/experimenting: 

Modeling process: What is the process involved with constructing a model? Is this a 
process that involves embodying alternative theories into models, testing those 
models - evaluating them, and thinking about their limitations and applications- and 
revising those models including deciding which model is most reasonable? 

Designing and creating models: How are models constructed? Are models designed for a 
purpose? Does the designer play a role in deciding what's important? How are 
attributes determined? Are attributes of the model exact copies of the object/process or 
abstracted features? What do you have to think about when making a model? How 
close does the model have to be to the thing itself? How do you know what's important to 
include? (ex: what parts of an atom do you have to model for modeling how atoms 
interact? all, some, etc.) 

Changing models: Would a scientist ever change a model? Why? When? 

3. Evaluation of models/theories/explanations: 

Model evaluation: Is there a way to decide whether one theory is better than another? If so, 
how does one decide if one model is better than another? When one evaluates a model, 
is one testing how workable that model really is or is one testing the underlying 
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assumptions of that model? 

Model criteria: What kinds of criteria are used to evaluate models? (These might 

include: accuracy, realism, validity; plausibility, consistency, utility for serving a 
given purpose; simplicity; elegance.) (This category is related to the purpose of 
modeling category,) Do ways of evaluating models change much over time? 

4. Purpose/Utility of modeling: 

Purposes of models: What are models for? (These might include: defining/developing 
theories, predicting/testing theories, refining theories, visualizing phenomena, and 
explaining phenomena.) Are models best used to verify reality or to develop and test 
ideas? What’s the best use of a model? 

Utility of models in science and science classes: How could you use models in science or 
science class? How might a scientist use a computer model like ThinkerTools? How 
could students use a computer model like ThinkerTools to learn science? 

Utility of multiple models: What’s the purpose of having multiple models of the same 
phenomena/object? How could they be used to test theories? 



Table 2: Dimensions of modeling knowledge 

We have assessed these dimensions of students’ modeling knowledge by using several 
instruments and methods of data collection. Those include (1) a pre and post modeling 
questionnaire, (2) students’ final research projects, and (3) retention interviews about modeling 
with twenty students from two classes two and a half months after the end of the curriculum. 

Please note that we conducted this research in the tradition of the design experiment with no strict 
control group for the model-enhanced curricula classrooms. As a result, we have determined the 
effect of the classroom curriculum by calculating pre/post curriculum gain of model 
understanding on the assessments as well as achievement of specific modeling knowledge. 

Our primary modeling assessment, the modeling questionnaire, included items about the 
nature, evaluation and purpose of models in various question formats. Those formats included 
among other types, a sorting task (circling all types of items that are models), enhanced multiple 
choice questions (“What is the best definition of a model and why?”), and enhanced true/false 
questions (“Could a scientist create an incorrect model and why?”). In our analysis, we calculated 
statistical differences in multiple choice items and coded some long answer responses in order to 
determine whether the responses improved. 

Student final research projects provide some indication of the presence and forms of 
student models. In order to analyze these projects, we coded types of student models students used 
in their research projects. 

The interview included questions about all dimensions of modeling and lasted between 
thirty to fifty minutes. It included contextualized questions about models and modeling related to 
students’ final projects (“Did you get a chance to try out the different rules for your research 
findings in the modeling step of the inquiry cycle? Why should a student do this?”), 
decontextualized questions about the nature of models and modeling (“In general, is any model 
just as good as another?” “Do scientists ever change or revise their models?”), and two activities, 
(“Here are two examples of scientific models of gravity. How would you decide which is the best 
model?” and “Suppose that you wanted to find out how long it takes for a student to get between 
classes at your school. Using the inquiry cycle, describe how you might investigate this 
question.”). We have transcribed half of the student interviews to determine student 
understanding of scientific modeling along the dimensions of model understanding as well as 
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how much information and knowledge students have retained from the curriculum. From these 
transcriptions, the first author summarized students' statements to several questions to the 
interviews along several of the dimensions of modeling knowledge. 

Results 



Knowledge about the nature of models 

There is clear evidence that students significantly increased their understanding of the 
nature of models from the model-enhanced version of ThinkerTools. In the first item of the 
written modeling assessment, for example, students were asked to circle items which they thought 
were models. Types of items ranged from a pencil or bicycle or a globe to scientific rules or 
theories. Analysis of data from all four of Teacher A classes on this question indicates that there 
was significant improvement on the students' understanding. For example, only 14% of students 
thought a causal rule was a model in the pre-test whereas 48% of them believed it was a model in the 
post-test ( %2 (1, 71=71) = 22.15, p < .001) . There were significant changes to students categorization 
of all the items that were models except items students already believed were models (like globes, 
a diagram of an atom, or a toy car), and no significant changes in other items such as the pencil, 
bicycle or tree, for which 80% of students agreed were not models. (See appendix for complete 
results) 

Our goal was to expand students understanding of a model beyond the typical meaning of a 
toy or small copy of an object, to a set of rules, representations, and reasoning structures that allow 
a person to predict and explain phenomena. While this assessment item assesses superficial 
understanding of model knowledge (what the word ‘model' means), it nonetheless gives some 
indication that students understanding of the concept became more sophisticated to include sets of 
rules, theories and simulations. Further, it is important to note that some students might have 
interpreted all objects in this assessment item as models since the objects were represented by 
simplified and idealized representations of themselves. Students would be correct in this 
interpretation. This sophisticated interpretation, however, occurred in relatively few cases 
(roughly 5-10%). 

An additional item on the modeling assessment that asks students what is the best 
definition of the word ‘model' from the point of view of building a scientific theory shows modest 
improvement. 30% of students in the pre-test chose one of the most sophisticated options, “a set of 
rules that allow you to predict and explain,” or “a simplified or idealized picture of something” 
compared to 46% in the post-test ( %2 (1, ti=61) = 4.17, p = .04). (see appendix for modeling question) 
It is interesting to note that even in the post-test, 18% of students still picked the final response, that 
they “really didn't know.” These responses indicate that there is clearly room for improvement in 
students' understanding the nature of a model and supports the literature that indicates modeling 
knowledge is difficult to obtain. 

Several other items on the modeling assessment about the nature of models do not add 
much additional information on student modeling knowledge gain because of a ceiling effect in 
the assessment item. For example, 94% agreed in the pre-test and post-test that there can be 
different models of the same thing. In other words, students understood the concept of multiple 
models. 86% of students in the pre-test and 92% of students in the post-test agreed that if they and 
their partner had different theories, they could have a computer programmer create two different 
computer models for each of their theories. Again, students understood the concept of multiple 
models. Analysis of students' long answers to this question about the purpose of multiple models 
showed great improvement, however. 53% of their responses improved towards answers such as “to 
compare each theory and see who's right” compared to 30% of responses that stayed the same and 
16% that got worse. Finally, 92% of students agreed in the pre-test and post-test that a scientist could 
create an incorrect model. Students have some constructed understanding that models do not have 




AERA 1998, San Diego 




page 12 



Scientific Modeling 



Christina Schwarz & Barbara White 



to be correct. One interpretation for the ceiling effects has been mentioned in the work of Gilbert. 
His research shows that replacing the words “law” or “theory” with the word “model” in 
assessment items moves students towards a constructivist stance in answering questions about the 
nature of science. 

There are two other modeling assessment items that investigate students’ ideas about the 
constructed nature of models. In other words, do students think models represent absolute reality 
or are they useful constructions? In one question, 83% of students in the pre-test agreed that “Even 
the best scientific theories and models aren’t necessarily true; they’re just ways of helping us 
understand the world” compared to 87% in the post-test ( %2 (1, n= 63) = 1.0, p = .31) (Long answer 
analysis shows no significant improvement in students’ responses). 

Additional evidence about students understanding of the nature of models comes from 
student project reports, and student interviews. Analysis of student projects from four teacher B 
classes indicates that after the curriculum, most students had a basic understanding of what a 
model is within the inquiry cycle. 88% of students that did a final project included some sort of 
model in their project report which means that they included something under the heading of 
model which was an interpretation that went beyond restating their data. Further, most of these 
models were in the form of a general purpose rule that often included some sort of explanation. For 
example, one student wrote, “Falling objects speed up as they fall. I think this is because gravity is 
always applying force on everything ... This rule was the closest I could find to my real world 
experience. Regardless of mass, the height dropped from or weight, the object will speed up, until 
confronted with an obstacle.” 

Student interviews several months after the curriculum ended show that students 
maintained a substantial understanding of the nature of models. Out of the currently coded nine 
students who participated in modeling interviews, students’ responses ranged from fairly 
unsophisticated, ‘a model is a diagram, picture, or object that can be shown,’ 4 to a medium level of 
sophistication, ‘a model is a rule that predicts something,’ to very sophisticated such as PL’s 
response, “Well, it can be a theory or rule about what you think happens in real life, or it can be a 
representation of something. Any representation of a real thing, like a car model, or a theory. It’s 
a representation of the real world.” The first author scored the students’ response based on how 
abstract their model was. Out of the nine currently coded student interviews, seven demonstrated 
medium to high levels of sophistication about the nature of a model. This success appears to have 
little relationship with previous academic achievement, teacher, or previous experience with the 
ThinkerTools curriculum. 

Knowledge about the modeling process 

Students did not make similar gains in their understanding of the process of modeling 
from the model-enhanced version of ThinkerTools. For example, one question on the written 
model assessment addressed how students think models are constructed. In the model assessment 
item, “If a scientist wanted to create a scientific model of an atom in order to predict how that atom 
will interact with other atoms, what parts of the atom would a scientist include in the model?” More 
students chose the response, “every single part of the atom” after the curriculum ( yl (3, n=71) = 

5.69, p = .13) (See table 4 below). 



4 When student responses are written within single parentheses, this indicates that their statements are not exact quotes but 
summaries of their statements. 
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Question 10. If a scientist wanted to create a scientific model of an 
atom in order to predict how that atom will interact with other 
atoms, what parts of the atom would a scientist include in the 
model? 


Pre-Test 


Post-Test 


A. Every single part of the atom. 


22 (31%) 


30 (42%) 


B. Only the main parts of the atom 


15 (21%) 


8 (11%) 


C. Only parts useful for predicting how it will interact with other 
atoms. 


34 (48%) 


33 (46%) 



Table 4 



While this item is somewhat difficult to interpret, two aspects should be noted. First, 
student did not construct computer models in this curriculum, and therefore can only reflect upon 
the types of models they have seen. As a result, the less sophisticated option A, became more popular 
in the post test. Secondly, many students chose option C overall, showing some degree of 
sophistication about the process of modeling. 

The other modeling assessment item that asks students about the nature of the modeling 
process stated, “Would a scientist ever change or revise a scientific model?” 87% of students in the 
pre-test stated yes, and 94% in the post-test agreed ( %2 (1, n = 68) = 2.27, p = .13). Long answer 
analysis to this questions shows no significant improvement in students' responses because of the 
ceiling effect on this question. 

Information about students' knowledge of the modeling process was difficult to obtain from 
the interview for several reasons. Students in the model-enhanced version of ThinkerTools 
constructed preliminary models and chose among models similar to their preliminary ones. 
Therefore, questions about the process involved with constructing the model is less relevant for 
students in this curriculum. Further, students' scientific research was always structured within 
the inquiry cycle. Therefore, understanding the modeling process of (1) embodying a theory into a 
model, (2) testing the model, and (3) revising that model is inherent to the ThinkerTools inquiry 
process and was difficult to test. As a result, questions relating to the students' modeling process 
are indirect and relate to what students understood of the modeling forms, and the purpose of the 
models. 

Nonetheless, student interviews indirectly indicate that two months after the curriculum 
finished, students maintained some understanding of the modeling process. To obtain this 
information, the first author scored the forms of student models from students' self-reports of final 
projects in the interview as well as models from their inquiry thought-experiment at the end of the 
interview. In scoring model forms, the first author expected to gain insight in students' 
understanding of the purpose and process of modeling. The student model forms ranged from 
fairly unclear or unsophisticated such as 'a model as a construction to show someone else to see if 
they thought it made sense,' to a more sophisticated form of 4 a model as a predictive/descriptive 
rule,' to one of the most sophisticated forms 'a model as a predictive explanatory rule or a 
predictive representation.' Overall, students showed a fair amount of sophistication about model 
forms. Seven of nine students showed at least a medium level of sophisticated in their modeling 
forms. Again, this success appears to have little relationship with previous academic 
achievement, teacher, or previous experience with the ThinkerTools curriculum. 
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Knowledge about the evaluation of models 

Students’ gains in their understanding of the evaluation of models from the model- 
enhanced version of ThinkerTools was mixed. In one item on the written modeling assessment, 
for example, students were asked whether they agree or disagreed with the following statement: 
“Since scientists disagree about why dinosaurs became extinct, it’s clear that no one understands 
exactly how it happened. Therefore, any scientific model or theory of how it happened is just as 
good as any other.” In the pre-test, 59% of students agreed compared to 67% of the students on the 
post-tests ( %2 (1, n=6 4) = 1.31, p = .25). We had hoped that the curriculum would encourage students 
to understand that this statement was false - that models are evaluated with criteria and that some 
have more value in certain situations than others. However, these students begin with a 
relativistic orientation 5 , and apparently, this item indicates that the curriculum further 
encouraged this orientation. In fact, long answer analysis shows significant change towards the 
relativist position. 56% of students’ reasoning in their long answers stayed the same while 25% 
decreased in sophistication towards answers like “If nobody knows how it happened, any answer 
could be true.” 

Student interviews two months after the curriculum elaborate students’ reasoning. When 
students were asked whether in general Ts any model as good as another?’ seven out of nine 
students said 'yes’ with answers ranging from unsophisticated notions that 'as long as you worked 
hard on your research, all models were all the same,’ to more sophisticated response of 'all models 
are equally good when there is no way to know which one is right.’ Both students with more 
sophisticated responses said, 'not all models are as good as any other’ 'because some people don’t 
conduct their research carefully and 'some models are less well-thought out than others.’ Since 
seven of the nine students answered that all models are as good as each other, most scored a 
medium-level of sophistication on this question, indicating that while they had explicit criteria 
with which to evaluate models in the curriculum, this did not help to distinguish among models, 
and students moved towards a relativist position. Again, this success appears to have little 
relationship with previous academic achievement, teacher, or previous experience with the 
ThinkerTools curriculum . 

Another item on the written assessment asked students whether they agreed with the 
statement, “When a scientists evaluates a scientific model, she looks for certain qualities such as 
how accurate and reasonable the model is.” 89% in the pre-test agreed compared to 95% in the post- 
test ( %2 (1, n= 62) = 1.6, p = .20). Long answer analysis for this question shows significant 
improvement towards having students mention criteria such as accuracy, plausibility, utility, 
and consistency. 68% improved their responses, and 27% of the responses remained the same 
quality. Finally, students were asked whether they agreed with the statement, “Ways of 
evaluating scientific models or theories don’t change much over time.” 38% of students agreed in 
the pre-test compared to 36% in the post-test. It is not clear what the students understood from this 
question since many of their long answers seemed to indicate that they interpreted the question as 
'models don’t change much over time.’ 

In initial analysis, students demonstrated somewhat less sophistication about model 
evaluation in the interviews, although results are not necessarily indicative of students’ overall 
understanding of model evaluation from the interview. Out of the nine interviews, three students 
could not remember any model evaluation criteria. (Future interview analysis will give more 
complete results of student understanding of model evaluation criteria.) Again, students 
responses ranged from not remembering any model evaluation criteria and thinking of 
evaluating a project by looking over the material’ to remembering that evaluation criteria existed 
and that model utility was important, to knowing that models had to be accurate and plausible. For 



5 A relativistic orientation is one in which a person believes that all claims have equal value since there is no absolute 
method of distinguishing among them. 
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example, when I asked PL what criteria he might use to evaluate his research findings, he stated, 
“Well, I looked if my data seemed like plausible, and I looked to see if my model could be used to 
predict anything really accurately.” Again, most students scored a medium or medium to poor 
level of sophistication for this question which in retrospect is not as informative as other items in 
the interview. As previously stated, further interview analysis will help to clarify these results. 

In videotaped student project presentations at the end of the curriculum, student project 
work also indicated some deeper level understanding of the process of modeling as well as the 
model evaluation criteria. The following example illustrates some of this understanding. A 
ThinkerTools student, JP, was presenting his project on gas/fluid resistance in order to answer 
the question, what is the motion of an object dropped through a thick fluid like honey. He and his 
partner had performed an experiment and analyzed their data which indicated that over a long 
period of time, the object slowed down in this very thick fluid. During his project presentation, he 
stated, 

We estimated the rule before we went onto the computer, and our rule was that, um, when an 
object is affected by friction, it slows down. So we went on the computer and we tried out that 
one. [the computer modeling rule 'slows down’] ... and the object slowed down so much that it 
didn’t even go all the way [fall to the bottom of the container] which wouldn’t happen in real 
life. So we tried um, ... the one where you could use it at a constant speed, and that seemed to 
work right, which puzzled us a little bit, because that’s not what our data showed. So we were a 
little inconsistent there. So, on our ratings at the end, our self-assessment, that kind of messed 
us up 'cuz ... we had to be careful. We weren’t exactly all-too careful, obviously, and the data 
showed that we messed up. 

JP’s real world data seem to indicate that the objects are slowing down. But, the computer model 
simulation shows that if the objects were slowing down, they would eventually stop. Since this 
doesn’t happen in the real world, Mark realizes that their real world analysis was incorrect, and 
that he had an inconsistency in this data that he needed to resolve. This student showed a 
sophisticated understanding of modeling in that he understood that computer modeling is useful 
for testing theories. 

Knowledge about the utility of models 



Finally, students make striking gains in their understanding of the utility of models 
from the model-enhanced version of ThinkerTools. The written modeling assessment provides 
the strongest evidence for students’ improvement in this understanding. For example, when 
students were asked the question, “From a scientific point of view, which is the best use of a 
model?” 41% of students in the pre-test chose the most sophisticated answer, D, “to develop and test 
ideas” compared to 60% in the post-test ( %2 (1, n=63) = 7.2, p = .007). In another evaluation question, 
students were asked, “If you were an astronomer trying to determine the path of a comet in our 
solar system, which of the following would you rather have? A. A scale model (a smaller version 
of our solar system), or B. A computer simulation or computer model of our solar system.” 71% of 
students in the pre-test picked B, the computer simulation. This compares to 83% of students 
choosing B in the post-test ( %2 (1, n=58) = 3.77, p = .052). 

Two additional long answer only questions on the modeling assessment showed 
significant improvement in students understanding the utility of models. In one question, “How 
could computer models help scientists with their research?” 63% of responses improved towards 
answers like, “it allows them to test things they normally couldn’t test like frictionless 
environments” or “they could test their theories” while 25% of the remaining responses remained 
the same quality. In the second question, “How could computer models help students learn 
science?” students showed some, although not as dramatic improvement. 50% of students 
improved their responses towards answers such as “ It helps them understand the science” or “it 
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lets them experiment” or even “it lets them test their ideas” compared to 40% of responses whose 
quality remained the same. 

Finally, interviews indicate that students demonstrated a fairly robust understanding of 
the purpose of the model design software in the curriculum, and of the utility of scientific models in 
general. When students were asked what the purpose was of the model design rules on the 
computer, responses ranged from the ambiguous response of ‘to show a better way of 
understanding in case they didn’t get it right/ to ‘to be able to conduct experiments not otherwise 
possible and choose the rules and see which one was the best/ to most sophisticated, ‘to show you 
what you picked so you could see which one looked better and to make you think about which one 
was right.’ In general, students showed a fair amount of sophistication in this question and 
averaged fell in between the categories medium and medium to high level of sophistication in 
their response. This result appears to have little relationship with previous academic achievement, 
teacher, or previous experience with the ThinkerTools curriculum. 

Secondly, when students were asked what they thought scientific models could be useful 
for, there was a great range of responses in the interview questions. An example of the least 
sophisticated response was ‘scientific models can be used for anything. They’re not useful for 
scientists, but for students learning science, they can help them learn to construct better models.’ A 
more typical response was ‘a scientific model can help explain what and why something happens. 
For scientists, they explain why things happen. For students learning science, models can be 
useful to help them visualize and understand.’ The most sophisticated response indicated that 
‘scientific models can help people find out information. They can help construct cars, and predict 
what they will be like. For scientists, scientific models can help them visualize and test things that 
are not otherwise possible such as DNA. For students learning science, scientific models can help 
them see other people’s models, as well as helping them visualize and manipulate the models.’ 

Summary of Results 

To summarize, analysis of all multiple choice items on the written modeling assessment 
for all Teacher A students indicates an improvement in student understanding of modeling. 
When all assessment items were scored and given the points for the sophistication of their 
responses, the overall pre-test mean was 62% compared to an overall post-test mean of 70% ‘correct’ 
(t 71 = 6.15, p < .001, a = .64). One third of the points for this assessment were derived from the first 
item on the assessment, the modeling categorization task, and a third of the remaining questions 
showed ceiling affects. Therefore, this gain particularly reflects students improvement in their 
understanding of the nature of models, and in the utility of models. 

Analysis of the long answers from the modeling assessment supports the multiple choice 
analysis showing large gains in student understanding of the utility of models for scientists and 
for learning science, the utility of multiple models, some gains in students’ understanding of the 
evaluation of models (what criteria to use), and an increase in students’ relativist orientation 
towards model evaluation (any model is as good as any other). 

Finally, initial analysis of student interviews shows that in general, students had a robust 
understanding of the nature of models, a strong understanding of the utility of models, and again 
a somewhat relativist orientation towards the evaluation of models. 

How do these results compare with those from Grosslight, Unger, Jay, and Smith (1991)? 
Initial analysis seems to indicate that after the model-enhanced version of ThinkerTools, there is 
some evidence that our seventh graders understood more about the nature and process of modeling 
than the seventh graders they surveyed. Certainly, in comparing the same interview questions, 
“What comes to mind when you hear the word ‘model’? Are there different kinds of models? What 
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are models for? Can you use models in science? Do you think scientists would ever have more than 
one model for the same thing? Would a scientist ever change a model?” our seventh graders 
responded with some sophistication. However, it is difficult to determine students’ epistemologies 
concerning the relationship between reality and models from the above questions, particularly 
since the main question about constructing models, “What do you have to think about when 
making a model?” was not relevant to our students and as a result, not asked. There is some 
evidence that our seventh graders had a mixture of constructivist, nearly relativist orientation 
towards the nature of science while at the same time, some strong empiricist views as well. Our 
seventh grade students could be categorizes as having reached Grosslight et al.’s level 2 or mixed 
level 1 and 2 understanding of models and the nature of modeling. In a level 2 understanding of 
the nature of models: 

the student now realizes that there is a specific, explicit purpose that mediates the way the 
model is constructed. Thus the modeler’s ideas begin to play a role, and the student is 
aware that the modeler makes conscious choices about how to achieve the purpose. The 
model no longer must exactly correspond with the real-world object being modeled. Real- 
world objects or actions can be changed or repackaged in some limited ways (e.g., through 
highlighting, simplifying, showing specific aspects, adding clarifying symbols, or 
creating different versions). However, the main focus is still on the model and the reality 
modeled, not the ideas portrayed per se. Further, tests of the model are not thought of as tests 
of underlying ideas but of the workability of the model itself. (Grosslight et al., 1991, p. 817- 
818) 



Future Directions 

We plan to conduct further analysis of our seventh graders’ model understanding to gain 
greater depth and insight into these interpretations. Further analysis will include coding 
remaining interviews and interview questions as well as coding the remaining research book 
data, long answer data analysis, and written model assessment data from teacher B’s classes (in 
order to compare teacher effects). We will also be conducting regression analysis on students 
previous ThinkerTools experience and academic achievement to clarify results. Further, we are 
pursuing analysis of individual student modeling profiles to track the progression of modeling 
knowledge. These results are likely to be informative. For example, we hypothesize that each 
student advanced essentially one level in their model understanding. In other words, a student 
who knew nothing of models at the beginning of the curriculum might have learned that a model 
can be a visual object that copies another. Similarly, a student who began by thinking of a model as 
a small copy or toy might have moved towards thinking of a model as an abstract representation. 
An alternative hypothesis for which there has been some initial evidence suggests that lower 
academic-achieving students gained more modeling knowledge than higher academically 
achieving students because they had a better understanding and appreciation for the purpose and 
process of the computer modeling in the curriculum. Perhaps running the computer modeling 
simulation was not self-evident for these lower-achieving student as it was for some narrowly- 
focused and higher-achieving students. Results are forthcoming. 



Conclusions 

While interactive computer models and modeling as a methodology of science are being 
advocated for use in science classrooms, “there is still a need to examine student understanding 
and use of models in general and the characteristic knowledge and misunderstandings they hold 
about models.” (Benchmarks for Scientific Literacy, Project 2061) Further, it is critical that 
students understand the very process of science with which they are involved, as well as their 
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overall understanding about the nature of science because modeling is one of the most important 
components of the scientific endeavor. 

This paper has examined how a curriculum focused on scientific modeling can foster 
student understanding of the nature of models and the process of modeling. In the model- 
enhanced ThinkerTools curriculum, we found that teaching this model-enhanced curriculum 
fostered student’s understanding of the nature and utility of models without promoting similar 
gains in students’ understanding of the process of creating and evaluating models. 

Future analysis about the evolution of students modeling knowledge as well as the 
analysis of the relationship between previous ThinkerTools experience, academic achievement 
and modeling knowledge gain will further clarify these results. Teaching students about models 
and the process of modeling is a challenging yet promising way to encourage their development of 
sophisticated epistemologies of science. Future research on refining model-oriented curricula 
and assessments as well as classroom trials will be essential to this progress. 
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Written Modeling Assessment Results 



Questions About Models 



Question 7. Circle all of the items which you think are models: 



a scientific theory like 
Einstein’s 
theory of relativity 



a pencil 



a computer simulation 
like SIM CITY 



Pre: 21% Post 52% 

p<.001 



Pre: 24% Post: 17% 
p < .15 



Pre : 52% Post 68% 
p<.05 




a globe or map 



Pre: 37% Post: 35% 
p = .85 



Pre: 83% Post: 87% 
p -.47 



Pre: 24% Post: 24% 

p = 1.0 



a rule like “roughly every 
twenty-four hours, the sun 
rises in the east and sets in 
the west because the earth 
rotates on its axis” 




an orange 



a video animation like the 
movie TOY STORY 



Pre: 14% Post 48% 

p<.001 



Pre: 23% Post: 21% 

p -.80 



Pre: 35% Post 52% 

p<.01 




an equation like Newton’s 
second law which says that force 
applied on an object is equal to the 
mass of that object times the 
object’s acceleration. (F = m x a) 




a compact disc 

Pre: 32% Post: 25% 
p - .25 



Pre: 21% Post 55% 

p<.001 



a diagram of an atom 

Pre: 76% Post: 82% 
p = .32 
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a set of diagrams and plans 
for a building or a room 




a toy car 




Pre: 81% Post: 86% 
p = .32 




a ThinkerTools 
simulation 



Pre : 76% Post: 77% 
p = .76 




a tree 



Pre: 48% Post: 58% 
p = .29 




a person who displays clothes 



Pre: 46% Post: 83% 

p <.001 



Pre: 14% Post: 14% 

p = 1.0 



Pre: 49% Post: 59% 
p = .14 



Question 9: The following choices are all definitions of the word ‘model’. From the point of view of 
building a scientific theory, which is the best definition of a model? (Please circle one response) 



(A) A small copy of an object (Pre: 37% Post: 31%) 

(B) A set of rules that allow you to predict and explain (Pre: 16% Post: 34%) 

(C) A simplified or idealized picture of something (Pre: 13% Post: 13%) 

(E) A set of plans for constructing a building or bridge 

(F) A mannequin or someone who displays clothes 

(G) I really don't know! (Pre: 26% Post: 18%) 
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Question 10. If a scientist wanted to create a scientific model of an atom in order to predict how that 
atom will interact with other atoms, what parts of the atom would a scientist include in the 
model? (Please circle one response) 

(A) every single part of the atom (Pre: 31% Post: 42%) 

(B) only the main parts of the atom (Pre: 21% Post: 11%) 

(C) only parts useful for predicting how it will interact with 

other atoms (Pre: 48% Post: 46%) 

(p = .13) 

Question 11: Can there be different kinds of models of the same thing? For example, are there different 
kinds of models for an atom? 

(Please circle one response) 

(A) YES (Pre: 93% Post: 92%) 

(B) NO 

Question 12: If you and your partner had different theories, do you think you could have a computer 
programmer create two different computer models for each of your theories? 

(Please circle one response) 

(A) YES (Pre: 86% Post: 92%) 

(B) NO 

If you think it’s possible to create two different computer models, explain how this might be 
useful. If you don’t think it is possible to create two different computer models, explain why 
it’s not possible. 

53% responses their quality (long answer analysis done with a chi-suqare fit of the uniform 
distribution for the sample, p < .001) 

Question 13: Could a scientist create an incorrect model? 

(Please circle one response) 

(A) YES (Pre: 92% Post: 92%) 

(B) NO 
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Question 14: Would a scientist ever change or revise a scientific model? 
(Please circle one response) 

(A) YES 

(B) NO 



(Pre: 87% Post: 94%) 
(p = .13) 



Why or why not? 

66% responses remained the same quality (long answer analysis done with a chi-sugare fit of 
the uniform distribution for the sample, P < .001) 

Question 15: Do you agree or disagree with the following statement? (Please circle one response) 

“Even the best scientific theories and models aren’t necessarily true; they’re just ways of 
helping us understand the world.” 



(A) Agree 
(B) Disagree 



(Pre: 83% Post: 87%) 



Explain your choice: 62% responses remained the same quality (long answer analysis done 
with a chi-suqare fit of the uniform distribution for the sample, p < .001) 



Question 16. Scientific models are: (Please circle one response) 

(A) real and useful (models represent absolute reality) 

(B) not necessarily real and useful (models don’t necessarily 
represent absolute reality) 

(C) not real and not useful (models don’t represent absolute reality) 



(Pre: 25% Post: 31%) 
(Pre: 70% Post: 68%) 



Questions About Evaluating Models 

Question 17: Do you agree or disagree with the following statement? (Please circle one response) 

“Since scientists disagree about why dinosaurs became extinct, it’s clear that no one 
understands exactly how it happened. Therefore, any scientific model or theory of how it 
happened is just as good as any other.” 



(A) Agree 

(B) Disagree 



(Pre: 59% Post: 67%) 
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Explain your choice: 56% responses remained the same quality. 25% get worse (long answer 
analysis done with a chi-suqare fit of the uniform distribution for the sample, p < .001) 



Question 18: Do you agree or disagree with the following statement? (Please circle one response) 

“When a scientist evaluates a scientific model, she looks for certain qualities such as how 
accurate and reasonable the model is. ” 

(A) Agree (Pre: 89% Post: 95%) 

(B) Disagree 

If you agreed, describe some additional qualities. If you disagreed, explain why a scientist 
does not evaluate a model with certain qualities. 

58% responses improved (long answer analysis done with a chi-suqare fit of the uniform 
distribution for the sample, p < .001) 

Question 19: Do you agree or disagree with the following statement? (Please circle one response) 

‘Ways of evaluating scientific models or theories don’t change much over time ” 

(A) Agree (Pre: 38% Post: 36%) 

(B) Disagree 



Questions About the Usefulness of Models 

Question 20. From the scientific point of view, which is the best use of a model? (Please circle one 
response) 

(A) to be a toy 

(B) to copy an object or process 

(C) to help someone construct an object 

(D) to develop and test ideas (Pre: 41% Post: 60%) 

(p < .01) 
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Question 21: Do you agree or disagree with the following statement? (Please circle one response) 

‘‘Computer models and simulations can help us understand things lik e the motion of a 
comet in space or traffic patterns in a city ” 

(A) Agree (Pr e: 90% Post: 90%) 

(B) Disagree 

Question 22. Two biology research groups have different models or theories about how a dangerous 
virus dike HIV) might replicate. How useful would it be for them to build and test computer 
models of each other’s theories? (Please circle one response) 

(A) Very useful 

(B) Somewhat useful 

(C) Not useful 

Question 23. If you were an astronomer trying to determine the path of a comet in our solar system, 
which of the following would you rather have? 

(Please circle one response) 

(A) A scale model (a smaller version of our solar system) 

(B) A computer simulation or computer model of our solar system (Pre: 71% Post: 83%) 

Question 24: How could computer models help scientists with their research? 

63% responses improved (long answer analysis done with a chi-suaare fit of the uniform 
distribution for the sample, p < .001) 

Question 25: How could computer models help students learn science? 

50% responses improved (long answer analysis done with a chi-suaare fit of the uniform 
distribution for the sample, p < ,001) 



(Pre: 56% Post: 61%) 
(Pre: 41% Post: 46%) 
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